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(57) ABSTRACT 

An Internet-based electronic shopping system hosted on a 
televisioD-set-top-box combination controlled by a remote 
control unit. The set-top box (STB) includes a tuner, an 
internal or external modem and/or cable modem, configured 
to communicate with an Intemet service provider over the 
appropriate transmission media. The STB also includes 
purpose-type application software such as voice recognition 
software and bar code recognition software to support an 
electronic shopping system. The STB further includes an 
external interface for connection to a multiplicity of external 
peripheral devices, such as an interphone, VCR, video 
camera, printer, and the like. The remote control unit 
includes a keypad, a microphone, an optional speaker; an 
optional digital camera, and an optional speaker. Data input 
to an Intemet shopping Web program accessed through the 
STB's Web browser may thus be made through the 
microphone, keypad, and optionally through the digital 
camera. If the remote control unit includes the digital 
camera, bar code informalioa may be scaimed by the remote 
control unit and transmitted to the STB for identifying an 
item to be purchased. The remote control unit is also capable 
of being used as a cordless telephone or videophone, and 
might also function as a handset for an interphone when the 
appropriate communication links are connected to the STB 
through its external interface. When the system is used as a 
videophone, videographic images of a party on the other end 
of the line is displayed on the TV screen via a pop-up 
window. 

33 Claims, 6 Drawing Sheets 
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INTERNET SHOPPING SYSTEM UTILIZING Icr so as to provide a central means for accessing those 

SET TOP BOX AND VOICE RECOGNITION devices. In this way, the user docs not have to learn other 

mechanisms for controlling the devices. Although cflforts 
have been made in the prior art to provide a universal 

FIELD OF TOE INVENTION 5 multimedia access device adapted for communicating mul- 

Hie present invention relates generally to electronic shop- timedia information between the device and various appli- 

ping systems, and more particularly, to an Internet shopping ca^o° specific modules as disclosed m U.S. Pat. No. 5,636, 

system hosted on a television-set-top-box combination 211 to NewUnetal., the prior art docs not disclose or suggest 

including a remote controller with voice recogniUon capa- operating a set-top box with interface means to other housc- 

bilities. ^^^^ devices via voice commands. 

BACKGROUND OF THE INVENTION SUMMARY OF THE INVENTION 

The expansion of personal computers (PCs) into homes of These and other objects are accomplished in accordance 

average consumers has also expanded the usage of the present invention by an electronic shopping system 

Internet by such consumers, and with it, has opened a new 15 ^^^^^^ ^ television-set-top -box combination controlled 

avenue of shopping, specifically, Internet shopping. ^y a remote control unit which provides additional func- 

Currently, computers are the main source of Internet access. tionaUties to the system 

"Tri?"'''*^"'°T^'°«*'ir''r''T'°^'^^ >» accordance with one aspect of the invention, the 

or find PC operation to be diffioilt^ For those people. ^^^^^^^ . /^^^^^ ^ 

alternate means must be made available to allow them to 20 ^ ^ microphone for providing 

reap the benefits of the Internet without having to interact ^^^^^ ^^^^ ^J^^ ^^^^ ^(.^p 

Wit a computer. comprises means for connecting to an Internet service 

To solve the above problem, efforts have been made m the provider and a Web browser configured to receive input data 

pnor art to provide Internet access through common house- f^^jj^ remote control unit. The Web browser generates an 

hold electronics equipment such as a television and set-top ^ audio-visual display on a display unit in response to the 

box combination. For instance, Phihps-Magnavox and Sony -^^^^ ^^^^ aUowing the user to effectuate the electronic 

market a set-top box with Internet capabiUties under the shooDine 

trademark WebTV®. U.S. Pat. No. 5,512,935 to Majeti et / a . .u . * f iu 

*J«u In a second aspect of the present mvention, the remote 

al., also discloses a set-top box connected to a home con- ^ , ^ . j- •* i • i • j- 

'„ , , , i_ „'TT-u u-i30 control unit comprises a digital visual unage recordmg 

tro tier and personal computer combination. The home based _ l- j * . *l ott^ t-u- 

Li . • . -^L I * * means for providing videographic data to the STB. Tnis 

equipment m Majetiis able lo communicate with an Internet u j . u » j ona 

^ f .1. t • . r 1.1 J- * u *■ allows bar code images to be transmitted to the STB for 

service provider through a vanety of cable distribution •■, -c - . ^ u j 

. . ,^ uj.ii, ^j- identifying an Item to be purchased, 

means mcluding a pubhc switched telephone network via a * ^ 

QjQ^gm In a third aspect of the present invention, a voice genera- 

Although the telcvisiop-set-top-box combination may be ^or coupled to the Web browser oonverts character data to 

easier to operate than a PC, it may nonetheless require a ^^^Pj^^ ^h^ ^""^^f ^^^^ is transmitted to a 

substantive amount of time and effort to leam the operations ^P^^*^^^ the remote control umt for providing audio outpu 

of the set-top box, especiaUy the operation of the remote «P^^<^| also allows the remote control umt 

controUer working the set-top box. The more the functions „ f« ^ ,^ "^^^^^ P^^^^ ^ ^ 

provided by the set-top box, the more keys there will be on ^t^rphone or videophone. 

the remote controller for the user to learn. The learning of aU In a fourth aspect of the present mvention, the STB 

the functions in the remote controller keypad might be a comprises an external interface with a household peripheral 

laborious task for the user. Even if the user has learned the «^evice such as an intercom, interphone, videophone, VCR, 

functions of aU the keys, the maneuvering between various video camera, printer, fax, and the like, 

keys on the remote controller during Intemet shopping is nncnDiunnM np the nu AU/iMrc 

inefficient and cumbersome. ^^^^^ DESCRIPTION OF THE DRAWINGS 

In addition, an optional wireless keyboard and/or a virtual piG. 1 is an exemplary, simplified, semi-schematic block 

keyboard is often used when alphabetical data or compli- diagram of an electronic shopping system including a tele- 

cated input is required. A virtual keyboard is a fiill keyboard vision set, a set-top box, and a remote control unit in 

display on a CRT screen which is operated by the remote accordance with the present invention; 

control unit by designating key positions with arrow and 2 is a simplified, semi-schematic illustration of an 

enter keys. The use of such wireless keyboards or virtual exemplary implementaUon of the remote control unit of 

keyboards makes data input more laborious for the user. pjQ j. 

The use and learning of remote control functions become 55 pj^^ 3 .3 ^ exemplary, semi-schematic functional block 

even more cumbersome when the user must leam to use ^^^^^ ^^^^^^ PJCS ^ 2; 

other keypads, buttons, switches, and/or dials to work other , • ^ .• ui 1 j- <• 

household devices such as the VCR, interphone, video ^ ^'G- 4 "^■Hfl?'!:' ^^^■^^i^^'^^^'' "ock diagram of 

camera, printer, fax, and so on. '°P ""'^ °^ 

Accordingly, there is a need for a remote controller which so '^^ arc exemplary flow diagrams of an 

is able to communicate with a set-top box by means other 1°'=™=' shoppmg transaction usmg the set-top box and the 

than the keypad. Because voice input is easier and more """"l »■> accordance with the present invention, 

efficient than keypad input, such remote controller should DETAILED DESCRIPTION OF THE 

have voice recognition capabilities to allow users to provide INVENTION 

oral commands during Intemet shopping. 6S 

The user should also have access to other household In general terms, the present invention is directed to an 

devices directly through the set-top box and remote control- Internet-based electronic shopping system hosted on a 
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tclcvisionsct- top-box combinatioQ controfled by a remote 
control unit which provides additional functionality to the 
system. The set-top box (STB) includes a tuner and audio 
visual output functions for television reception capability. 
The tuner is configured to communicate with an Internet 5 
service provider over either satellite or through a cable 
medium and the STB also includes a modem capability for 
Intemet service provider communication over a telephone 
subscriber line interface. In addition to its conventional 
functions, the STB includes an external interface such as an 
[EEE1394 interface, USB, RS232C, SCSI, and the like, for 
connection to a multiplicity of external peripheral devices, 
such as an interphone, VCR, video camera, printer, and the 
like. 

The STB is configured to provide a user with Internet 
access in a manner such as currently provided by Web TV 
and further includes purpose- type application software such 
as voice recognition software and bar code recognition 
software to support an electronic shopping system. AppU- 
cation programs are either hosted on and loaded £rom a mass 
storage media such as a bard disk drive or are downloaded 
from an external server via a telephone modem connection, 
a cable or satelhte connection and/or wireless broadcast 
means. 

The remote control unit is configured to include a con- ^ 
ventional keypad and further includes a microphone, an 
optional speaker, an optional digital camera, and a provision 
for wireless transmission capability in order to send data to 
the STB. When the STB is configured with voice recognition 
software, an Internet shopping Web program or HTML text ^ 
may be accessed through the STB's Web browser and data 
input may be made by either voice, the remote control unit's 
keypad, a virtual keyboard displayed on the CRT, or by an 
optional wireless keyboard. Voice input is recognized by the 
voice recognition software and recognized data is trans- 
ferred to the Internet shopping Web program. 

When the remote control unit is configured to include a 
digital camera, bar code information may be scanned by the 
remote and bar code video data may be transmitted to the 
STB and recognized as bar code data by bar code recogni- 
tion software. Merchandise information is transmitted to a 
store's web server via any one of the Internet Hnks described 
above. 

When the remote control unit is configured with 
microphone, speaker and digital camera, it is also able to be 45 
used as a cordless telephone or video phone and might also 
function as a handsel for an interphone when the appropriate 
communication links are connected to the set top box 
through its external interface. When the system is used as a 
videophone, videographic images of a parly on the other end 50 
of the line may be displayed on the TV screen via a pop-up 
window. 

Considering the foregoing summary of the features of the 
system and method of the present invention, FIG. 1 depicts 
a simplified, semi-schematic block diagram of an exemplary 55 
electronic shopping system including a STB 10, television 
12, and remote control unit 14. The STB 10 receives 
television signals from satellite 16, over-the-air broadcast 18 
(also referred as aerial broadcast), or via cable through a 
cable provider 20, for performing conventional television 60 
reception functions. The STB also conununicates with an 
Internet Service Provider (ISP) 22 over the satellite 16 
medium or via cable through the cable provider 20. In 
addition, the STB 10 allows for ISP 22 communication over 
a telephone subscriber line 24 interface. 55 

Various peripheral devices 26 are also controlled through 
the STB 10. Such devices might include an interphone, 
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VCR, video phone, printer, fax, and the like. The centralized 
control of such devices 26 through the STB 10 helps 
facilitate their access and management. 

The television 12 set is coupled to the STB 10 and permits 
audio and display functions to the system. The display 
function is provided by the television screen and may 
include display of television programs, HTML pages, and 
video images created by the peripheral devices 26. The 
audio function is provided by one or more speakers 28 either 
resident within the television 12 set, or externally coupled to 
the television set. The television 12 unit may be an analog 
TV, a digital TV, or the like. 

The remote control unit 14 is also in communication with 
the STB 10 and provides additional functionalities to the 
system. FIG. 2 is an exemplary illustration of the remote 
control unit 14 and FIG. 3 is its functional block diagram in 
an exemplary, semi-schematic form. As in a conventional 
remote controller, the remote control unit 14 illustrated in 
FIG. 2 includes a keypad 30 for allowing input of keypad 
data to the STB 10. The keypad 30 in the described embodi- 
ment comprises a power key, varioxis numeric or alpha 
character keys, function keys (menu key, clear key, enter 
key, etc.), and arrow keys. A user depresses the keys on the 
keypad 30 to specify a particular operation the user wants to 
perform. For instance, up and down arrow keys along with 
the enter key might be used to select data displayed on the 
television screen. 

The remote control unit 14 also includes a microphone 32 
for capturing voice data upon an utterance by the user. Thus, 
a user may provide oral commands to the system during 
Internet shopping instead of keypad commands, making it 
easier and more pleasant for an average user to use the 
system. The use of voice recognition, imlike other situations, 
should be more efficient and more accurate for Internet 
shopping because it entails simple selection operations 
which require the utterance of only short phrases for recog- 
nition purposes. 

In an alternative embodiment, the remote control unit 14 
also includes a speaker 34 for output of voice messages to 
the user upon receipt of voice data from the STB 10. Such 
voice messages may include operation guidance, error 
messages, and the like. The speaker 34 can be operational 
either alone or in conjunction with the television speakers 
28. Alternatively, the speaker 34 in the remote control unit 
14 can be disabled and the television speakers 28 used in its 
place. The inclusion of the speaker 34 along with the 
microphone 32 also allows the remote control unit 14 to 
operate as a handset for cordless phone, interphone, and/or 
video phone functions that might be available through the 
STB 10. 

A digital visual image recording means such as a digital 
video camera 36 is also incorporated into the remote control 
unit 14 for providing additional functionahties to the system 
to promote easier Internet shopping. The camera 36 might 
comprise either gray scale or color video data. Preferably, 
the video image data will be in color to adequately represent 
the user's personal appearance, i.e., hair color, clothing 
color, and the like, when the camera 36 is tised for video 
phone or interphone purposes. Although the camera is 
described as a digital video camera 36, it should be under- 
stood by those having skill in the art that an apparatus for 
capturing still images rather than a full-motion video image 
can be used to implement the system according to the 
invention. Accordingly, the camera 36 might comprise a 
digital still camera, a video camera or any other type of 
device that outputs a digital image. 
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In addition, the camera 36 is preferably mounted on a for extracting the video images. From there, the video 
hinge on one end of the remote control unit 14 for allowing images might be submitted to a video signal processor (not 
the camera 36 to be rotated at a 90 or 180 degree angle. The shown) for converting the extracted video images to video- 
rotation of the camera not only facilitates video capture for graphic data to be di^layed on the display unit, 
video phone or interphone functions, but also eases the 5 If the remote control unit 14 receives only one type of data 
capture of barcode images when ordering groceries or other from the STB 10 (e.g. voice data or video data) as is 
barcoded items via the Interact, as described in further detail described in the current embodiment, the channel decoder 
IjgjQ^ 52 is no longer necessary for deciphering the type of dau 

_'. .n^-, • ./L. received given that the inputted data will always be of the 

Referrmg now to FIG. 3, user mput 6om the keypad 30, ^^j^ ^^^^ -^^^^^ ^^^^^ ^^^^^'^ 52 may be 

microphone 32, or digital camera 36 is processed by the 10 ^^-^^^^^ ^^^^^^ ^^^^j^j ^4 

remote control unit 14 before transmission to the STB 10. nO. 4 is an exemplary, semi-schematic block diagram of 

Input from the keypad 30 is sent to a keypad data encoder ^ { ^ comprises a tuner 82 and 

40 for translation of the keypad data mto a keypad code. ^ ^^^^^ ^^^^^^ ^4 conventional television 

Specifically, when a user depresses a key on the keypad 30, reception capabilities. The tuner 82 is further configured to 

the keypad data encoder 40 encodes the depressed keypad 15 p^^vide Internet connection with a Web server 72 via the ISP 

position information to a binary keypad code, such as an 42, through the satellite 16 medium or via cable through the 

ASCn character data or a hexadecimal code fortransmission cable provider 20. The STB 10 is also equipped with a 

to the STB 10. modem (not shown) for communicating with the Web server 

Input from the microphone 32, on the other hand, is sent 72 via the ISP 22, through telephone subscriber lines 24. 

to an analog to digital converter 42 for converting voice data ^ The Web server 72 provides various software programs 

from analog data to digital data according to well known 76, such as HTML pages, Internet shopping Web programs, 

principles. Video images captured by the digital camera 36 and other application software programs to a Web browser 

are sent to a video signal processor 44 for generating 70 through the Internet connection. The software programs 

videographic data also according to well known principles. 76 may be downloaded and stored in a local storage 74 unit. 

The input data from the three input media, i.e. the keypad ^ The local storage 74 may suiubly comprise a hard disk drive 
code from the keypad data encoder 40, voice data fi-om the or drive array, a PCMCIA programmable memory card, a 
analog to digital converter 42, and videographic data from high-capacity flexible media cartridge and floppy drive 
the video signal processor 44, are then sent to a channel combination, ROM, FROM (Flash ROM), SRAM or 
coder 46 in the remote control unit 14. The channel coder 46 DRAM memory chips, or the like. In addition to down- 
combines the input data from the three input media by a loaded software programs, the local storage 74 may also 
predetermined coding method and transmits the coded data store electronic program guide data including scheduling 
to the STB 10 through a transmitter 48. The predetermined information of television programs, as well as miscellaneous 
coding method allows the STB 10 to recognize and decode user-specific information, such as the user's e-mail address, 
the data received from the different input media of the ^5 shopping list information, personal notes, etc. 
remote control unit 14, The coding method, according to one The local storage 74 may further store software and data 
embodiment, entails placing a header and a delimeter for for a voice recognizer, video data recognizer, voice 
demarcating the beginning and end of particular types of generator, remote control unit driver, Web Browser, and any 
input data. In this scenario, the channel coder 46 transmits other type of appUcation. These types of software and data 
the input data along with the corresponding header and ^ can be pre-loaded, downloaded, and/or modified. Modifica- 
delimeter, for decoding by the STB 10. tion can be made by downloading or loading the software 

The transmitter 48 in the remote control unit 14 allows for and/or data from a local medium such as an IC card, a floppy 

wireless transmission of the coded data to the STB 10. hifira disk, or the like. 

Red (IR), radio frequency (RF), Ultra High Frequency The STB 10 also includes a remote control unit driver 58 

(UHI^, or any other wireless technology may be used for the 45 having an STB receiver 60, STB channel decoder 62, STB 

wireless transmission. If IR technology is iised, the trans- transmitter 64, and STB channel coder 66. The STB receiver 

mitter 48 may generally be equipped with an LED. If RF or 60 receives input data from the remote control unit 14 via 

UHF technology is used, the transmitter 48 may generally be wireless means. In an alternative embodiment, the STB 

equipped with an antenna or the like. receiver 60 also receives keyboard data from a wireless 

According to one embodiment, the remote control unit 14 50 keyboard 68. The wireless keyboard 68 is operational 

also includes a receiver 50 for wireless receipt of data from ^^F, or any other wireless technology 

the STB 10. The data is then sent to a channel decoder 52 for applicable to the remote control unit 14. 

decoding and extracting the data received for a particular STB receiver 60 submits the received input data to 

medium. In the illustrated embodiment, the remote control the STB channel decoder 62 for decoding the multiple 

unit 14 is equipped to receive voice data from the STB 10. 55 medium data to a keypad code, voice data, and/or video- 

The voice data, once extracted by the channel decoder 52, is graphic data. According to one embodiment, the STB chan- 

transmitted to a digital to analog converter 54 and the °el decoder 62 examines the header preceding the input data 

converted analog voice data is sent to the speaker for for identifying the input mediimj of the data. The diannel 

generating corresponding sound waves to the user. It should decoder then extracts the data between the header and the 

be appreciated by those having skill in the art, that necessary 60 delimiter and transmits the extracted data for further 

variations can be made to the remote control unit without processing, if required, based on the identified medium, 

departing from the spirit and scope of the invention should During Internet topping, if the identified medium is the 

other types of data other than voice data be sent by the STB keypad 30, the keypad code is extracted by the STB channel 

10 to the remote control unit 14. For instance, if the remote decoder 62 and submitted to the downloaded software 

control unit 14 is equipped with a display unit (not shown), 65 program 76 through the Web browser 70. 

the STB 10 might send video images to the receiver 50 and If the identified medi\im is the microphone 32 or digital 

the receiver 50 might submit them to the channel decoder 52 camera 36, the extracted voice data or videographic data 
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undergoes further processing, \bice data is submitted to a 
voice recognizer 78 configured with voice recognition soft- 
ware for converting the voice data to character data accord- 
ing to a number of conventional voice recognition algo- 
rithms. 5 

In the alternative embodiment, voice recognizer is 
included in the remote control unit. In this case, the remote 
control unit converts the voice data to character data and 
transfers the character data to the set top box. 

Vidcographic data is submitted to a video data recognizer 
80 which, in the described embodiment, is configured with 
barcode recognition software. The barcode recognition soft- 
ware further includes pattern recognition and barcode 
decode capabihtics for deciphering barcode images inputted 
to the system via the digital camera 36 (FIG. 3). Most 
retailers today attach a Universal Product Code (UPC) 
which is printed or tagged on each item of merchandise to 
be sold, as a barcode. Conventionally, the barcode might 
include an item's Stock Keeping Unit (SKU) code as well as 
other additional information pertaining to a specific item of ^ 
merchandise. The barcode data is converted to a UPC or 
SKU code by the barcode recognition software and trans- 
mitted to a downloaded purchase transaction program 
through the Web browser 70. With this capabiUty, a user can 
transmit an order for an item via the Internet by simply ^ 
transmitting the barcode image of the item. The barcode 
image should be available to the user if the user has the 
packaging of the item on band. Barcode data may also be 
available in catalogues or in old receipts from retailers 
offering the desired merchandise. ^ 

Id an alternative embodiment, the video data recognizer is 
included in the remote control unit. The remote control unit 
recognizes captured bar code image data and converts it to 
a bar code, such as UPC, SKU or the like, and transfers the 
bar code data to the set-top box. 

It should be noted, at this point, that those skilled in the 
art will appreciate various other codes, indicia, text, and the 
like, being used instead of the barcode for item identification 
purposes. Thus, the barcode recognition software may suit- ^ 
ably be modified, without departing from the spirit or scope 
of the invention, to decipher any other types of item iden- 
tifiers. 

In an alternative embodiment, the visual data recognizer 
80 is configured with other types of image recognition ^5 
software, such as an optical character recognition (OCR) 
program. If the visual data recognizer 80 is configured with 
an OCR program, handwritten notes (e.g. a handwritten 
shopping list) can be converted into text character codes for 
either input to the downloaded software program 76 through jq 
the Web browser 70, or for storage in the local storage 74 
unit for later use. 

A downloaded software program 76 can therefore receive 
input from the remote control unit 14 and/or the wireless 
keyboard 62. In addition, the downloaded program 76 has 55 
access to the local storage unit 74, an IC card interface 88 
unit, and an external interface 90, if needed. Details of the 
IC card interface 88 unit and the external interface 90 are 
provided below. 

The dovraloaded software program 76, in response to eo 
input data, outputs graphic and character display data 
through the Web browser 70 for conveyance to an audio 
visual output unit 84. The audio visual output unit 84 
transfers the graphic and character display data to the 
television 12 for display thereon. 55 

The audio data from the downloaded software program is 
transferred to a voice generator 86 configured with a voice 
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generating program that converts the audio data to digital 
voice data- The digital voice data is transferred to the STB 
channel coder 66 which, as the channel coder 46 (FIG. 3) in 
the remote control unit, combines all the output data by a 
predetermined coding method and transmits the coded data 
to the speaker 34 in the remote control unit 14 through the 
STB transmitter 64. Although the STB 10 of FIG. 4 only 
illustrates voice data as being output to the remote control 
unit, a person skilled in the art, as described above in 
conjunction with FIG. 3, should appreciate other types of 
data being transmitted to the remote control imit 14. 
However, in the instance where only one type of data is 
capable of being output to the remote control unit 14, the 
channel coder 68, like the channel decoder 52 (FIG. 3) in the 
remote control imit, becomes unnecessary and may be 
eliminated if desired. 

The voice data created by the voice generator 86 may 
optionally be transferred to the television speakers 28 
through the audio visual output unit 84. The television 
speakers 28 may be made operational alone or in conjunc- 
tion with the remote control speakers 34. 

In addition to the above, the STB 10 includes an IC card 
interface 88 configured to read information from and write 
information to an IC, or smart card. The IC card and IC card 
interface 88, in combination, provide a suitable means for 
authenticating an STB 10 as a valid receiver of particular TV 
services. The IC card can also provide secured payment 
methods (credit card, prepaid electronic cash, etc.), storage 
of personal informarion, personal identification, and the like. 
While referred to as an IC card, the card is a personal 
memory card or data card which looks and feck much like 
an ordinary credit card. The IC card may be either contact 
based or contactless. The simplest contact-type card might 
be a magnetic tape storage stripe affixed in a particular 
location on its reverse side. Alternatively, a contact or 
contactless IC card may comprise a microprocessor, an 
electrically erasable field-programmable read-only memory 
(EEPROM), a Flash ROM (FROM) and, optionally, cir- 
cuitry for inductively receiving an RF power signal. In its 
second configuration, the IC card is capable of capadtively 
transferring data signals between the card and the IC card 
interface 88 and may be interrogated, programmed or repro- 
grammed by read/write circuitry comprising the card inter- 
face 88. 

An additional I/O device is provided in the STB 10 in the 
form of an external interface 90 for centralized access of 
various peripheral devices 26 (FIG. 1). The external inter- 
face 90 allows communication between the peripheral 
devices 26 and the STB 10. As a result, a two-way exchange 
of information is made possible between the peripheral 
devices 26 and the STB 10. This information can include 
video signals, data signals, voice signals, audio signals, 
image signals, and control signals. The external interface 90 
might comprise an IEEE 1394 interface, USB, RS232C, 
SCSI (small computer system interface), LAN, IrDA, and 
the like. The peripheral devices 26 might comprise an 
intercom, interphone, VCR, video camera, video phone, 
printer, fax machine, and the like. 

As an example, to accomplish interphone functionalities 
through the STB 10, an interphone outside the user's front 
gate should include video-camera capabilities. The inter- 
phone is connected to the STB 10 through the external 
interface 90. Control of the interphone is made by an 
interphone application software in the STB 10. The apph- 
cation software might be loaded from the local storage 74 
unit, downloaded from the Web server 72 through the Web 
browser 76, or preloaded into the STB 10. The videographic 
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picture captured by the interphone outside the user's door is the server program retrieves informadon corresponding to 

displayed on the television 12 via a pop-up window, \foicc the selected item from a Price Lookup (PLU) Table. In the 

capabilities are provided by the microphone 32 in the remote described embodiment, all merchandise information is 

control unit 14. Audio capabilities arc provided by the maintained in the PLU Table. The PLU Table is, in turn, 
remote control speaker 34, television speakers 28, or both. 5 stored and maintained in the Web server 72 database. The 

It can be appreciated, therefore, that access to the interphone ^LU Table suitably comprises a merchandise specific mfor- 

through the STB 10 fadUtates such access by allowing the ^a^ioo set. The merchandise specific mformation set may be 

user to receive an interphone call from a visitor outside the ^^^^^ i° ^ ^^^^^V °{ ^^X^' ^""^ ^ '"of advantagcotisly 

home without having to move away from a television configured as scqucnUd cntnes, with each enl^ speci^^^ 

^ , L - L * u- T jj** *i_ a particular piece of merchandise. A merchandise entry 
program that he or she might be watchmg. In addition, the lo ^^^^ include the store's SKU number which is identified to 

user IS spared from havu^ a separate mtcrphone receiving ^ particular produa's UPC by a suitable conversion routine, 

unit smce the television-STB combinaUon, along with the ^ merchandise entry might further include a text string 

remote control umt 14, acts as the receivmg umt. g-^jjjg brand or trade name of the product and including 

Referring now to FIGS. 5A and 5B, operation of the a generic description of the product. In addition to the SKU 

electronic shopping system of the present invention is dis- number and identifying string, each item entry also might 

cussed in detail. An Internet shopping transaction starts by include a price field. 

selecting a Web browser function 100 fi-om a menu of Upon review of the selected merchandise information, 
functions available through the STB 10. The menu is dis- and after a decision to purchase the merchandise, the user 
played on the television screen and selection is made via the proceeds to payment processing 127. The user has the option 
keypad 30 on the remote control unit 14. Alternatively, 20 of using the IC card for making payment for the item 128. 
selection may be made through the microphone 32 by orally If card is to be used, the card is processed 130 by the 
voicing the selection. Selection of the Web browser function IC card interface 88, The processing might include extract- 
causes display of the Web browser 70 on the television set. ^ customer ID number stored in the card and transmitting 
The user then selects a Web site from the browser 102 by I^ to the Web server 72 for determining whether the user 
direcUy keying-in a web site address or selecting a site from ^ ^, ^ preferred customer participating in a preferred customer 
a menu of available Internet shopping sites. Hie menu may P^^^ ^^j"^^ "^'^ allow hmi or her reward points or dis- 
be customized by the user based on his or her favorite counted prices. The purchase amoi^^^ 
r ^ . , . a * 1- . f *u T * * to the user s account for being billed at a later time. 
Internet shopping sites, or may reflect a list of the Internet „„ . , • . , j ^ 
* ,1 . . *u Whether or not the IC card is used as a method of 
sues most recently accessed by the user. , \\ / j • V ^_ - rT-.^i 
^ , . . 30 payment, the purchase order information can be prmted 132 
Once a shoppmg Internet site has been specified, the Web a printer coupled to the STB 10 through the external 
server downloads the corresponding client purchase trans- interface 10 for record-keeping purposes, if a printer is 
action program and/or HTML text 104 to the STB through connected to the set top box 10. 

the Web browser. In addition needed programs such as a Accordingly, there has been brought to the art of elec- 

voice recognition program, video data recogmtion program ^^^^ ^ ^ ^^^^ ^ ^ ^^^^^^ ^^^^ 

voice generating program and IC card mterface program, if j^^^^^^^ ^ ^ ^^-^^ ^ television-set-top^ox com- 

not akeady preloaded m the STB, are b^^^ bination in conjunction with a remote control unit with voice 

storage unit 74 or downloaded from the Web server 72. recognition capabilities. Internet shopping is fadUtated by 

The downloaded cUent purchase transaction program dis- aUowing users to input voice commands through the micro- 
plays an initial display on the television screen along with ^ phone in the remote control unit. The optional camera also 

introductory visual and/or audio guidance, and requests for facilitates Internet shopping by allowing users to simply 

user selection of a purchase item 106. The audio guidance is j^put the barcode image of the item to be purchased. The 

effectuated through the remote control speaker 34, television external interface in the STB and the optional speaker in the 

speakers 28, or both. remote control unit also faciUtate the operation of peripheral 

If keypad input is detected 108, the corresponding keypad 45 devices by providing a centralized access to those devices, 

code is extracted by the STB channel decoder and trans- while the invention has been described with respect to 

ferred to the transaction program 110 through the Web particular illustrated embodiments, those skilled in the art 

browser. If microphone input is detected 112, voice recog- and technology to which the invention pertains will have no 

nition is performed by converting the voice data to the difficulty devising variations which in no way depart from 

corresponding character data. The extracted charaaer data is the scope and spirit of the present invention. For example, 

then transferred to the transaction program 116. Detection of vs^hile the illustrated embodiments have been described in 

a camera input 118 causes, in the illustrated embodiment, the connection with a set-top box, it will be appreciated that the 

running of the barcode recognition program 120. In alter- present system and method may be devised within the 

native embodiments, other video data recognition programs television set itself, vwthout the need of a separate set-top 

such as an OCR program may be run if appUcable. Barcode 55 Accordingly, the present invention is not limited to the 

recognition causes the conresponding UPC or SKU numeric specific embodiments described above, but rather, is defined 

data to be transferred to the transaction program as an input by the scope of the appended claims. 

122. If no input is received, or an erroneous input is What is claimed is: 

received, an error message or operation guidance is provided 1 . In an electronic shopping system including a television 
124 to the user via the television screen and speakers 28, ^ set coupled to a set-top box, the set-top box being configured 

and/or via the remote control speaker 34. to access the Internet through a Web browser resident in the 

The data input process continues until all necessary set-top box, the set-top box further being configured to 

selections have been made by the user to complete an item communicate with a remote control unit, the remote control 

selection 126. unit comprising: 

The client purchase transaction program in the STB 10 is 65 a first input medium including a keypad for providing 

in communication with the server purchase transaction pro- keypad data to a shopping application program 

gram on the Web server 72. Upon client selection of an item, accessed through the Web browser; 
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a seooDd input medium including a microphone for pro- 
viding voice data to the shopping application program 
accessed through the Web browser; and 

a transmitter coupled to the first input medium and the 
second input medium for transmitting the keypad data 5 
and the voice data to the shopping application program 
as input for effectuatiog a purchase. 

2. The remote control unit of claim 1 further comprising 
a third input medium including a digital visual image 
recording means for providing videographic data, the third 
input medium being coupled to the transmitter for transmit- 
ting the videographic data to the set-top box. 

3. The remote control unit of claim 1 further comprising: 
a receiver for receiving output voice data firom the set-top 

box; and 

a speaker coupled to the receiver for generating sound 
waves corresponding to the received output voice data. 

4. The remote control unit of claim 1 further comprising 
a voice recognizer for converting voice data to character 
data, the character data being transmitted by the transmitter 

to the set-lop-box. 20 

5. In an electronic shopping system including a television 
set and a remote control unit configured with a microphone, 
the remote control unit providing input data to a set-top box 
coupled to the television set, the set-top box comprising: 

a receiver for receiving the input data from the remote 25 
control unit, the input data including voice data from 
the microphone; 

a voice recognizer for converting voice data to character 
data; 

means for accessing a shopping application program; 

means for providing the input data to a shopping appli- 
cation program for effectuating a purchase; and 

an audio-visual output unit coupled to a Web browser for 
generating an audio-visual display on a display unit in 
response to the input data. 

6. The set-top box of claim 5 further comprising: 

a voice generator coupled to the Web browser for con- 
verting character data to output voice data; and 
a transmitter for transmitting the output voice data. ^ 

7. The set-top box of claim 5 further comprising an 
external interface with a household peripheral device. 

8. The set-top box claim 5 further comprising an interface 
with a customer ID card. 

9. The set-top box of daim 5 further comprising a bcal 
storage unit for storing and retrieving application software 
programs. 

10. The electronic shopping system of claim 5, wherein 
the display unit is a television monitor. 

11. An electronic shopping system comprising: 
a remote control unit comprising: 
at least two input media for providing input data, the first 

input medium including a keypad for providing keypad 
data, and the second input medium including a micro- 
phone for providing voice data; and 55 

a transmitter coupled to the input media for transmitting 
the input data; 

a set-top box in communication with the remote control 
unit, the set-top box comprising: 

a receiver for receiving the input data from the trans- 60 

mitter in the remote control unit; 
means for accessing a shopping application program; 
means for providing the input data to the shopping 

application program for effectuating a purchase; and 
an audio -visual output unit coupled to the Web browser 65 

for generating an audio -visual display on a display 

unit in response to the input data. 
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12. The electronic shopping system of daim 11, wherein 
the remote control unit further comprises a third input 
medium including a digital visual image recording means 
for providing videographic data to the set -top box. 

13. The electronic shopping system of claim 12, wherein 
the remote control unit comprises means for bar code 
rccognitioa 

14. The electronic shopping system of claim 12, wherein 
the set-top box comprises means for bar code recognition. 

15. The electronic shopping system of daim 11, wherein 
the set-top box further comprises: 

a voice generator coupled to the Web browser for con- 
verting character data to output voice data; and 
a transmitter for transmitting the output voice data; and 
the remote control unit further comprises: 

a receiver for receiving the output voice data; and 
a speaker coupled to the receiver for generating sound 
waves corresponding to the received output voice 
data. 

16. The electronic shopping system of claim 11, wherein 
the set- top box further comprises an external interface with 
a household peripheral device. 

17. The electronic shopping system of claim 11, wherein 
the set-top box further comprises an interface with a cus- 
tomer ID card. 

18. The electronic shopping system of claim 11, wherein 
the set-top box is coupled to a local storage unit for storing 
and retrieving application software programs, 

19. The electronic shopping system of claim 11, wherein 
the display unit is a television monitor. 

20. The electronic shopping system of claim 11 further 
including a wireless keyboard system for communicating 
with the set-top box. 

21. The electronic shopping system of claim 11, wherein 
the remote control unit is further configured with a speaker 
and is capable of functioning as a wireless phone. 

22. The electronic shopping system of claim 11, wherein 
the remote control unit is further configured with a speaker 
and is capable of functioning as a handset for an interphone. 

23. The electronic shopping system of claim 11, wherein 
the remote control unit is further configured with a speaker 
and is capable of functioning as a handset for a videophone. 

24. An elearonic shopping method comprising the steps 
of: 

accessing a data communications network through a set- 
top box; 

accessing a shopping application program fi'om the data 

commtmications network; 
entering input data through a remote control unit having 

at least a keypad for entering keypad data and a 

microphone for entering voice data; 
converting the voice data to charaaer data; 
transmitting the input data from the remote control unit to 

the shopping application program for effectuating a 

purchase; 

receiving audio-visual data from the shopping application 

program in response to the input data; and 
displaying the audio- visual data on a display unit. 

25. The method of claim 24, wherein the step of entering 
input data comprises the step of entering videographic data 
through the remote control unit further including a digital 
visual image recording means. 

26. The method of claim 25, wherein the videographic 
data comprises bar code data. 

27. The method of claim 24 further comprising the steps 
of: 
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receiving character data from the shopping application 
program; 

converting the character data to output voice data; 
transmittiag the output voice data to a remote control unit 

having a speaker; and 
generating sound waves corresponding to the received 

output voice data. 

28. The method of claim 24 further comprising the step of 
interfacing with a household peripheral device through the 
set-top box. 

29. The method of claim 24 further comprising the step of 
interfacing with a customer ID card through the set-top box. 

30. The method of claim 24 further comprising the step of 
interfacing with a local storage unit for storing and retriev- 
ing application software programs. 

31. The method of claim 24 further comprising the step of 
entering input data through a wireless keyboard system. 

32. The remote control unit of claim 1, wherein the second 
input meditmi further provides voice data to a plurality of 
devices coupled to the set-top box for interacting with the 
plurality of devices via voice from the remote control unit. 
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33. In an electronic shopping system including a televi- 
sion set and a remote control unit configured with a plurality 
of input media types, the remote control unit providing input 
data to a set-top box coupled to the television set, the set-top 
box comprising: 

a receiver for receiving the input data from the remote 
control unit, the input data being generated by the 
plurality of input media types; 
a decoder for decoding the input data into a plurality of 
individual data sets, each individual data set being 
associated with a particular input media type, the 
decoder determining whether ftirthcr processing is 
needed for a particular individual data set based on its 
associated input media type; 
means for accessing a software application program; 
means for providing the input data to the software apph- 

cation program; and 
means for receiving an output from the software apphca- 
tion program in response to the input data. 
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